Overcoming spatio-angular trade-off in light field acquisition using compressive sensing

In contrast to conventional cameras which capture a 2D projection of a 3D scene by integrating the angular domain, light field cameras preserve the angular information of individual light rays by capturing a 4D light field of a scene. On the one hand, light field photography enables powerful post-capture capabilities such as refocusing, virtual aperture, depth sensing and perspective shift. On the other hand, it has several drawbacks, namely, high-dimensionality of the captured light fields and a fundamental trade-off between spatial and angular resolution in the camera design. In this paper, we propose a compressive sensing approach to light field acquisition from a sub-Nyquist number of samples. Using an off-the-shelf measurement setup consisting of a digital projector and a Lytro Illum light field camera, we demonstrate the efficiency of the compressive sensing approach by improving the spatial resolution of the acquired light field. This paper presents a proof of concept with a simplified 3D scene as the scene of interest. Results obtained by the proposed method show significant improvement in the spatial resolution of the light field as well as preserved post-capture capabilities

Compressive Sensing System for Light Field Reconstruction

Svjetlosno polje \engl{light field} je zapis snimane scene koji osim intenziteta piksela pruža i informaciju o smjeru upada zraka svjetlosti na fotosenzor kamere. Dodatne informacije o sceni omogućuju veći spektar tehnika analize slike nakon snimanja, uključujući i detekciju dubine. Glavni nedostatak svjetlosnog polja je mala rezolucija finalne slike. Sustav opisan u ovom radu predlaže rješenje problema niske rezolucije slike korištenjem sažimajućeg očitavanja \engl{compressive sensing} kojim se iz malog broja mjerenja može značajno podići rezolucija finalne slike. Osim sažimajućeg očitavanja, koristila se i transportna matrica svjetla kao alat za implementaciju dualnog vida. Također, opisana je i detekcija dubine pomoću fundamentalne matrice i mape dispariteta. Za akviziciju svjetlosnog polja korištena je Lytro Illum kamera koja koristi polje mikroleća između glavne velike leće i fotosenzora kako bi sačuvala informaciju o smjeru upadu zraka svjetlosti. U sustavu se koristio i visokorezolucijski projektor za projiciranje kalibracijskih i mjernih uzoraka na scenu. Opisana su dva eksperimenta, jedan za rekonstrukciju sažimajućim očitavanje, drugi za detekciju dubine. Prikazani su rezultati eksperimenata i navedeni su prijedlozi za poboljšanja njihovih rezultata. Spomenuti prijedlozi predlažu nove konfiguracije scene, optimalnu poziciju kamere i spajanje dvaju eksperimenata u jedan. Takvo spajanje rezultiralo bi detekcijom dubine u visokorezolucijskim snimkama svjetlosnog polja.Light field is a record of a scene that, in addition to the pixel intensity, also provides information on the direction of the light beam on the camera's photosensor. Additional scene information allows a wider range of image post processing, including depth detection. Main disadvantage of light field images is low resolution of the final image. The system described in this thesis suggests a solution to the low image resolution problem by using compressive sensing, which can significantly increase the resolution of the final image from a small number of measurements. In addition to compressive sensing, the light transport matrix was used as a tool for dual photography implementation. Also, depth detection is described using a fundamental matrix and disparity map. The Lytro Illum camera was used to capture the light field. It uses the microlens field between the main lens and the photosensor to acquire information about the direction of the light beam. Also, a high-resolution projector was used in the system for projecting calibration and measurement samples on the scene. Two experiments are presented, one for compressive sensing reconstruction, another for depth detection. Results of both experiments are presented and commented alongside suggestions for results improvement. These suggestions include new scene configurations, optimal camera position, and merging two experiments into one. New system would result in depth detection from high-resolution light field images

Instantaneous frequency estimation using the Hilbert transform

Hilbertova transformacija je operator koji omogućuje jednostavno dobivanje amplitude i faze ulaznog signala koristeći analitički signal. Analitički signal je zapis u kojem je realna komponenta ulazni signal, a imaginarna komponenta Hilbertova transformacija signala. Koristeći testne signale i programski alat Matlab, u prvom eksperimentu se pokazalo kako pomoću analitičkog signala možemo pratiti trenutnu frekvenciju ulaznog signala, te su rezultati eksperimenta popraćeni grafičkim prikazom izlaznih signala. U drugom eksperimentu algoritam za izračunavanje analitičkog signala implementiran je na ARM Cortex M3 procesor. Implementirani algoritam radi na principu računanja Brze Fourierove transformacije, izjednačavanja dijela spektra koji odgovara negativnim frekvencijama s nulom i računanja Inverzne brze Fourierove transformacije takvog spektra. U procesoru se računa i faza signala, a izlazni signal je frekvencija ulaznog signala koja se učita i crta u Matlabu. Gledajući dobivene rezultate i uspoređujući ih s onima iz Matlaba, može se vidjeti kako algoritam za ARM procesor estimira trenutnu frekvenciju signala, što dokazuje da je algoritam uspješno implementiran.Hilbert transform is an operator that allows simple calculation of the amplitude and phase of the input signal using the analytic signal. Analytic signal is a signal whose real part is the input signal and its imaginary part is Hilbert transform of the input signal. Using programming tool Matlab in first experiment, it was shown how the frequency of the input signal can be monitored using the analytic signal. The results were displayed as graphs that represent frequency of the input signal. In second experiment, the algorithm for calculating the analytic signal was implemented on ARM Cortex M3 processor. Implemented algorithm calculates Fast Fourier transform of the input signal, replaces coefficients that correspond to negative frequencies with zeros, and calculates Inverse fast Fourier transform. Algorithm calculates phase of the signal and the output signal from the processor is signal frequency which is imported into Matlab and drawn. Looking at the results of the experiment and comparing them with the results calculated in Matlab, it’s apparent that ARM algorithm estimates instantaneous signal frequency, which proves that the algorithm is successfully implemented

Compressive Sensing System for Light Field Reconstruction

Svjetlosno polje \engl{light field} je zapis snimane scene koji osim intenziteta piksela pruža i informaciju o smjeru upada zraka svjetlosti na fotosenzor kamere. Dodatne informacije o sceni omogućuju veći spektar tehnika analize slike nakon snimanja, uključujući i detekciju dubine. Glavni nedostatak svjetlosnog polja je mala rezolucija finalne slike. Sustav opisan u ovom radu predlaže rješenje problema niske rezolucije slike korištenjem sažimajućeg očitavanja \engl{compressive sensing} kojim se iz malog broja mjerenja može značajno podići rezolucija finalne slike. Osim sažimajućeg očitavanja, koristila se i transportna matrica svjetla kao alat za implementaciju dualnog vida. Također, opisana je i detekcija dubine pomoću fundamentalne matrice i mape dispariteta. Za akviziciju svjetlosnog polja korištena je Lytro Illum kamera koja koristi polje mikroleća između glavne velike leće i fotosenzora kako bi sačuvala informaciju o smjeru upadu zraka svjetlosti. U sustavu se koristio i visokorezolucijski projektor za projiciranje kalibracijskih i mjernih uzoraka na scenu. Opisana su dva eksperimenta, jedan za rekonstrukciju sažimajućim očitavanje, drugi za detekciju dubine. Prikazani su rezultati eksperimenata i navedeni su prijedlozi za poboljšanja njihovih rezultata. Spomenuti prijedlozi predlažu nove konfiguracije scene, optimalnu poziciju kamere i spajanje dvaju eksperimenata u jedan. Takvo spajanje rezultiralo bi detekcijom dubine u visokorezolucijskim snimkama svjetlosnog polja.Light field is a record of a scene that, in addition to the pixel intensity, also provides information on the direction of the light beam on the camera's photosensor. Additional scene information allows a wider range of image post processing, including depth detection. Main disadvantage of light field images is low resolution of the final image. The system described in this thesis suggests a solution to the low image resolution problem by using compressive sensing, which can significantly increase the resolution of the final image from a small number of measurements. In addition to compressive sensing, the light transport matrix was used as a tool for dual photography implementation. Also, depth detection is described using a fundamental matrix and disparity map. The Lytro Illum camera was used to capture the light field. It uses the microlens field between the main lens and the photosensor to acquire information about the direction of the light beam. Also, a high-resolution projector was used in the system for projecting calibration and measurement samples on the scene. Two experiments are presented, one for compressive sensing reconstruction, another for depth detection. Results of both experiments are presented and commented alongside suggestions for results improvement. These suggestions include new scene configurations, optimal camera position, and merging two experiments into one. New system would result in depth detection from high-resolution light field images